-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use BCryptGenRandom instead of RtlGenRandom on Windows. #84096
Conversation
(rust-highfive has picked a reviewer for you, use r? to override) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me. rust-random/getrandom#65 (comment) was a helpful summary of the situation and justification for BCryptGenRandom.
Equivalent change removing RtlGenRandom in the getrandom
crate after dropping XP support: rust-random/getrandom#177.
@bors r+ |
📌 Commit b8a9112 has been approved by |
🌲 The tree is currently closed for pull requests below priority 1000. This pull request will be tested once the tree is reopened. |
…olnay Use BCryptGenRandom instead of RtlGenRandom on Windows. This removes usage of RtlGenRandom on Windows, in favour of BCryptGenRandom. BCryptGenRandom isn't available on XP, but we dropped XP support a while ago.
@bors r- Failed in #84405:
https://github.com/rust-lang-ci/rust/runs/2403918836#step:24:30801 |
I think we just need to add the appropriate bcrypt flags here: rust/src/test/run-make-fulldeps/tools.mk Line 104 in 673d0db
|
☔ The latest upstream changes (presumably #85594) made this pull request unmergeable. Please resolve the merge conflicts. |
Finished benchmarking commit (c102653): comparison url. Summary: This benchmark run did not return any relevant changes. If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf. @rustbot label: -perf-regression |
Used to be needed when Rust sometimes relied on RtlGenRandom instead of BCryptGenRandom, but that changed as of rust-lang/rust#84096 advapi32 is still used in Rust core, but only in the deprecated `env::home_dir` API, which we don't use: - https://github.com/rust-lang/rust/blob/22f8bde876f2fa9c5c4e95be1bce29cc271f2b51/library/std/src/sys/windows/c.rs#L681-L689 - https://github.com/rust-lang/rust/blob/22f8bde876f2fa9c5c4e95be1bce29cc271f2b51/library/std/src/sys/windows/os.rs#L292 - https://github.com/rust-lang/rust/blob/22f8bde876f2fa9c5c4e95be1bce29cc271f2b51/library/std/src/sys/windows/os.rs#L319 - https://github.com/rust-lang/rust/blob/22f8bde876f2fa9c5c4e95be1bce29cc271f2b51/library/std/src/env.rs#L566-L575
std depends on Bcrypt on Windows starting with rust 1.57. See rust-lang/rust#84096. Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
std depends on Bcrypt on Windows starting with rust 1.57. See rust-lang/rust#84096. Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Due to rust-lang/rust#84096 we need to link to bcrypt on windows else we get linker errors. cc influxdata/flux#4406 https://app.circleci.com/pipelines/github/influxdata/flux/11766/workflows/d1ef0c61-babb-4682-8c74-7087013c873e/jobs/38341 ``` C:\\Users\\circleci\\AppData\\Local\\go-build\\pkgconfig\\windows_amd64\\lib/libflux-3dd4b4346280d5a830d03b58d109be24cf44fd0ab5964e0113b470e7ba1223ed.a(std-c90c2a9f63fc5684.std.04e7289e-cgu.0.rcgu.o): In function `std::sys::windows::rand::hashmap_random_keys': /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40\/library\std\src\sys\windows/rand.rs:10: undefined reference to `BCryptGenRandom' /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40\/library\std\src\sys\windows/rand.rs:10: undefined reference to `BCryptGenRandom' collect2.exe: error: ld returned 1 exit status Exited with code exit status 2 ```
Rust 1.57 added the dependency: rust-lang/rust#84096
Issue rust-lang#84096 changed the hashmap RNG to use BCryptGenRandom instead of RtlGenRandom on Windows. Mozilla Firefox started experiencing random failures in env_logger::Builder::new() (Issue rust-lang#94098) during initialization of their unsandboxed main process with an "Access Denied" error message from BCryptGenRandom(), which is used by the HashMap contained in env_logger::Builder The root cause appears to be a virus scanner or other software interfering with BCrypt DLLs loading. This change adds a fallback option if BCryptGenRandom is unusable for whatever reason. It will fallback to RtlGenRandom in this case. Fixes rust-lang#94098
Make HashMap fall back to RtlGenRandom if BCryptGenRandom fails With PR rust-lang#84096, Rust `std::collections::hash_map::RandomState` changed from using `RtlGenRandom()` ([msdn](https://docs.microsoft.com/en-us/windows/win32/api/ntsecapi/nf-ntsecapi-rtlgenrandom)) to `BCryptGenRandom()` ([msdn](https://docs.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptgenrandom)) as its source of secure randomness after much discussion ([here](rust-random/getrandom#65 (comment)), among other places). Unfortunately, after that PR landed, Mozilla Firefox started experiencing fairly-rare crashes during startup while attempting to initialize the `env_logger` crate. ([docs for env_logger](https://docs.rs/env_logger/latest/env_logger/)) The root issue is that on some machines, `BCryptGenRandom()` will fail with an `Access is denied. (os error 5)` error message. ([Bugzilla issue 1754490](https://bugzilla.mozilla.org/show_bug.cgi?id=1754490)) (Discussion in issue rust-lang#94098) Note that this is happening upon startup of Firefox's unsandboxed Main Process, so this behavior is different and separate from previous issues ([like this](https://bugzilla.mozilla.org/show_bug.cgi?id=1746254)) where BCrypt DLLs were blocked by process sandboxing. In the case of sandboxing, we knew we were doing something abnormal and expected that we'd have to resort to abnormal measures to make it work. However, in this case we are in a regular unsandboxed process just trying to initialize `env_logger` and getting a panic. We suspect that this may be caused by a virus scanner or some other security software blocking the loading of the BCrypt DLLs, but we're not completely sure as we haven't been able to replicate locally. It is also possible that Firefox is not the only software affected by this; we just may be one of the pieces of Rust software that has the telemetry and crash reporting necessary to catch it. I have read some of the historical discussion around using `BCryptGenRandom()` in Rust code, and I respect the decision that was made and agree that it was a good course of action, so I'm not trying to open a discussion about a return to `RtlGenRandom()`. Instead, I'd like to suggest that perhaps we use `RtlGenRandom()` as a "fallback RNG" in the case that BCrypt doesn't work. This pull request implements this fallback behavior. I believe this would improve the robustness of this essential data structure within the standard library, and I see only 2 potential drawbacks: 1. Slight added overhead: It should be quite minimal though. The first call to `sys::rand::hashmap_random_keys()` will incur a bit of initialization overhead, and every call after will incur roughly 2 non-atomic global reads and 2 easily predictable branches. Both should be negligible compared to the actual cost of generating secure random numbers 2. `RtlGenRandom()` is deprecated by Microsoft: Technically true, but as mentioned in [this comment on GoLang](golang/go#33542 (comment)), this API is ubiquitous in Windows software and actually removing it would break lots of things. Also, Firefox uses it already in [our C++ code](https://searchfox.org/mozilla-central/rev/5f88c1d6977e03e22d3420d0cdf8ad0113c2eb31/mfbt/RandomNum.cpp#25), and [Chromium uses it in their code as well](https://source.chromium.org/chromium/chromium/src/+/main:base/rand_util_win.cc) (which transitively means that Microsoft uses it in their own web browser, Edge). If there did come a time when Microsoft truly removes this API, it should be easy enough for Rust to simply remove the fallback in the code I've added here
Make HashMap fall back to RtlGenRandom if BCryptGenRandom fails With PR rust-lang#84096, Rust `std::collections::hash_map::RandomState` changed from using `RtlGenRandom()` ([msdn](https://docs.microsoft.com/en-us/windows/win32/api/ntsecapi/nf-ntsecapi-rtlgenrandom)) to `BCryptGenRandom()` ([msdn](https://docs.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptgenrandom)) as its source of secure randomness after much discussion ([here](rust-random/getrandom#65 (comment)), among other places). Unfortunately, after that PR landed, Mozilla Firefox started experiencing fairly-rare crashes during startup while attempting to initialize the `env_logger` crate. ([docs for env_logger](https://docs.rs/env_logger/latest/env_logger/)) The root issue is that on some machines, `BCryptGenRandom()` will fail with an `Access is denied. (os error 5)` error message. ([Bugzilla issue 1754490](https://bugzilla.mozilla.org/show_bug.cgi?id=1754490)) (Discussion in issue rust-lang#94098) Note that this is happening upon startup of Firefox's unsandboxed Main Process, so this behavior is different and separate from previous issues ([like this](https://bugzilla.mozilla.org/show_bug.cgi?id=1746254)) where BCrypt DLLs were blocked by process sandboxing. In the case of sandboxing, we knew we were doing something abnormal and expected that we'd have to resort to abnormal measures to make it work. However, in this case we are in a regular unsandboxed process just trying to initialize `env_logger` and getting a panic. We suspect that this may be caused by a virus scanner or some other security software blocking the loading of the BCrypt DLLs, but we're not completely sure as we haven't been able to replicate locally. It is also possible that Firefox is not the only software affected by this; we just may be one of the pieces of Rust software that has the telemetry and crash reporting necessary to catch it. I have read some of the historical discussion around using `BCryptGenRandom()` in Rust code, and I respect the decision that was made and agree that it was a good course of action, so I'm not trying to open a discussion about a return to `RtlGenRandom()`. Instead, I'd like to suggest that perhaps we use `RtlGenRandom()` as a "fallback RNG" in the case that BCrypt doesn't work. This pull request implements this fallback behavior. I believe this would improve the robustness of this essential data structure within the standard library, and I see only 2 potential drawbacks: 1. Slight added overhead: It should be quite minimal though. The first call to `sys::rand::hashmap_random_keys()` will incur a bit of initialization overhead, and every call after will incur roughly 2 non-atomic global reads and 2 easily predictable branches. Both should be negligible compared to the actual cost of generating secure random numbers 2. `RtlGenRandom()` is deprecated by Microsoft: Technically true, but as mentioned in [this comment on GoLang](golang/go#33542 (comment)), this API is ubiquitous in Windows software and actually removing it would break lots of things. Also, Firefox uses it already in [our C++ code](https://searchfox.org/mozilla-central/rev/5f88c1d6977e03e22d3420d0cdf8ad0113c2eb31/mfbt/RandomNum.cpp#25), and [Chromium uses it in their code as well](https://source.chromium.org/chromium/chromium/src/+/main:base/rand_util_win.cc) (which transitively means that Microsoft uses it in their own web browser, Edge). If there did come a time when Microsoft truly removes this API, it should be easy enough for Rust to simply remove the fallback in the code I've added here
This removes usage of RtlGenRandom on Windows, in favour of BCryptGenRandom.
BCryptGenRandom isn't available on XP, but we dropped XP support a while ago.